Traditional market research to date has mostly relied on verbal input and ratings. However, advances in neurobiology have confirmed that people are largely emotional, subconscious (intuitive) decision-makers. Armed with this new knowledge it is imperative for market research to monitor, gauge, capture and gain insights from emotions, as opposed to the traditional emphasis on capturing rationally-oriented, cognitively-filtered input from test participants. How exactly to capture the emotional input, and then what to make of it, poses a huge dilemma for market researchers and major corporate clients struggling to understand how to “read” the emotional data in order to make better decisions related to massive advertising expenditures, new product launches, store environments, packaging, brand positioning, and a host of other market research type issues.
Of all the various psychological or neurobiological means of capturing emotional data, facial coding is the single most attractive option. Facial coding attempts to read the facial expressions of individuals to gain insight into their emotional responses. One reason for the appeal of facial coding, is the longstanding tradition of documentation in this area. Charles Darwin originated the practice, studying not only his own children's facial expressions but those of others. Darwin found that even a person born blind has the same facial expressions as a non-blind individual. In other words, facial expressions are universal and innate (from birth) ways of expressing emotions. Additionally, facial expressions also represent the only place in the body where the muscles attach right to the skin. Another key advantage of facial coding over other psycho-physiological methods is that it is tangible, on the surface, with results evident to the naked eye. This is in contrast to fMRI brain scans or EEG methods, both of which are more invasive, costly, and difficult to execute. These later methods involve a “black box” approach of reading physiological data based on either blood flow or electrical impulses within the brain.
The modern facial coding pioneer is Paul Ekman, an academic who with his partner, Wally Friesen, created the Facial Action Coding System (“FACS”). FACS uses identification of the contraction and/or relaxation of various facial muscles to attempt to determine an individuals emotions. FACS defines 32 Action Units (“AU”s) which represent a contraction or relaxation of one or more facial muscles. These AUs can then be processed to assign an emotion based on the AUs observed. The assignment of AUs to emotions is based upon well documented research into the link between emotion and facial muscle movements. FACS is fully described in Ekman, P., Friesen, W. V., Facial Action Coding System: A Technique for the Measurement of Facial Movement (also known by its acronym of FACS), Consulting Psychologists Press, Palo Alto, Calif. (1978), which is hereby incorporated by reference in its entirety herein.
Several methods exist for reading the facial muscles of an individual when exposed to a stimulus to assign AUs. A first method is to apply sensors to the skin of the face to capture precise readings of muscle movements. A second method is utilizing trained observers who observe the subject's reaction to the stimulus in real time or later by watching a video record of the subject. A third method is to employ automated methods that utilize computer technology to automatically generate AU codings for facial muscle activity.
Such automated methods detect frontal faces in the video stream and code each frame with respect to a variety of facial muscle movements. Such methods could include algorithms based on AdaBoost, support vector machines, and linear discriminant analysis, as well as feature selection techniques. One example technique would be to select a subset of Gabor filters using AdaBoost and then train support vector machines on the outputs of the filters selected by AdaBoost. Another example method may be to utilize “virtual” dots which are placed over a participant's face on a computer screen. Subsequent movements of facial muscles are then tracked by a computer as movements of these dots.
Facial muscle capture and coding techniques are described in detail in U.S. Pat. No. 7,113,916 issued Sep. 26, 2006 and U.S. Pat. No. 7,246,081 issued Jul. 17, 2007, both entitled “Method of Facial Coding Monitoring for the Purpose of Gauging the Impact and Appeal of Commercially-Related Stimuli,” both issued to Applicant, and both of which are hereby incorporated by reference in their entireties. Capture and coding techniques and analysis is also described in pending U.S. patent application Ser. No. 11/062,424 titled “Computerized Method of Assessing Consumer Reaction to a Business Stimulus Employing Facial Coding,” filed Feb. 20, 2005 by Applicant and U.S. patent application Ser. No. 11/491,535 titled “Method and Report Assessing Consumer Reaction to a Stimulus by Matching Eye Position with Facial Coding,” filed Jul. 21, 2006 by Applicant, both of which are hereby incorporated by reference in their entireties.
While FACS is generally accepted as the “gold standard” within the field, other less sophisticated or detailed methods exist for coding facial coding activity to gather emotional data. One of these less sophisticated methods, which will be hereinafter referred to as the “Simplified” method, consists of taking a simplified topline assessment approach to how a particular emotion reveals itself on the face based on as little as one to three places to look for generalized muscle movement, i.e., near the eyes/eyebrow region, near the mouth, or chin, or in one case the nose, and deriving from cruder generalized findings of basic movement in those areas how a person is feeling. While not limited to such an example, an example of this method is diagrammed in FIG. 1.
Various embodiments of the present disclosure are independent of the method of coding utilized, whether FACS or Simplified, and is also independent of whether the coding is done automatically or manually as previously described.
The accuracy of using facial muscles to discover an individual's emotions depends on how the raw data is translated into emotions, and how those emotions are used to draw conclusions. Specifically, in the FACS system, the accuracy of the system depends largely on how the AUs are mapped to emotions and how those emotions are then analyzed to produce meaningful results. The analysis portion can be tricky because facial coding data can be captured on a split-second basis, down to 1/30th of a second, and in the case of using FACS the analysis involves numerous facial muscles and approximately 20 or more action units (or AUs). As a result, the goal of extracting the most optimal, insightful and decisive data from facial coding can present difficult challenges. Even with the Simplified method of detecting and recording facial muscle activity, identifying the data that is more meaningful than other data to help in understanding in a meaningful way consumers' reactions to various forms of advertising/marketing, product/service innovations, new packaging, brand positioning, and responses to the acting talent that is or may appear in advertising, etc., represents both a substantial problem and a tremendous opportunity for companies that want to learn best from the data. Thus, there exists a need in the art for an accurate scoring means for deriving emotions from the captured raw data of facial coding.